<div class="chunk" id="custom_load"><div class="rcode"><div class="error"><pre class="knitr r">## Error in library(kernlab): there is no package called 'kernlab'
</pre></div>
</div></div>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
  <!--
  Design by Free CSS Templates
http://www.freecsstemplates.org
Released for free under a Creative Commons Attribution 2.5 License

Name       : Emerald 
Description: A two-column, fixed-width design with dark color scheme.
Version    : 1.0
Released   : 20120902

-->
  <html xmlns="http://www.w3.org/1999/xhtml">
  <head>
  <meta name="keywords" content="" />
  <meta name="description" content="" />
  <meta http-equiv="content-type" content="text/html; charset=utf-8" />
  <title>Using Your Own Model in train</title>
  <link href='http://fonts.googleapis.com/css?family=Abel' rel='stylesheet' type='text/css'>
  <link href="style.css" rel="stylesheet" type="text/css" media="screen" />
  </head>
  <body>
  <div id="wrapper">
  <div id="header-wrapper" class="container">
  <div id="header" class="container">
  <div id="logo">
  <h1><a href="#">Using Your Own Model in train</a></h1>
</div>
  <!--
  <div id="menu">
  <ul>
  <li class="current_page_item"><a href="#">Homepage</a></li>
<li><a href="#">Blog</a></li>
<li><a href="#">Photos</a></li>
<li><a href="#">About</a></li>
<li><a href="#">Contact</a></li>
</ul>
  </div>
  -->
  </div>
  <div><img src="images/img03.png" width="1000" height="40" alt="" /></div>
  </div>
  <!-- end #header -->
<div id="page">
  <div id="content">

<h1>Contents</h1>  
<ul>
  <li>
  <a href="#Introduction">Introduction</a>
  </li>
  <li>
  <a href="#Illustration1">Illustrative Example 1: SVMs with Laplacian Kernels</a>
  </li>
  <li>
  <a href="#Components">Model Components</a>
  </li>
  <li>
  <a href="#Illustration2">Illustrative Example 2: Something More Complicated - LogitBoost</a>
  </li>
  <li>
  <a href="#Illustration3">Illustrative Example 3: Nonstandard Formulas</a>  </li>
  </li>
  <li>
  <a href="#Illustration4">Illustrative Example 4: PLS Feature Extraction Pre-Processing</a>      
  </li> 
  <li>
  <a href="#Illustration5">Illustrative Example 5: Optimizing probability thresholds for class imbalances </a>      
  </li>   
 </ul>   
  
<div id="Introduction"></div>
<h1>Introduction</h1>  
<p>
The package contains a large number of predictive model interfaces. However, you may want to create your own because:
</p>
<ul>
  <li> you are testing out a novel model or the package doesn't have a model that you are interested in
  </li>
  <li> you would like to run an existing model in the package your own way
  </li>
  <li> there are pre-processing or sampling steps not contained in the package or
you just don't like the way the package does things
  </li>
</ul>
<p>
You can still get the benefits of the <a href="http://cran.r-project.org/web/packages/caret/index.html"><strong>caret</strong></a>  infrastructure by creating your own model.
</p>
<p>Currently, when you specify the type of model that you are interested in (e.g. <tt><span class="hl std">type</span> <span class="hl kwb">=</span> <span class="hl str">&quot;lda&quot;</span></tt>), the  <span class="mx funCall">train</span> function runs another function called <span class="mx funCall">getModelInfo</span> to retrieve the specifics of that model from the existing catalog. For example:
</p>
<div class="chunk" id="custom_modelInfo"><div class="rcode"><div class="source"><pre class="knitr r"><span class="hl std">ldaModelInfo</span> <span class="hl kwb">&lt;-</span> <span class="hl kwd">getModelInfo</span><span class="hl std">(</span><span class="hl kwc">model</span> <span class="hl std">=</span> <span class="hl str">&quot;lda&quot;</span><span class="hl std">,</span> <span class="hl kwc">regex</span> <span class="hl std">=</span> <span class="hl num">FALSE</span><span class="hl std">)[[</span><span class="hl num">1</span><span class="hl std">]]</span>
<span class="hl com">## Model components</span>
<span class="hl kwd">names</span><span class="hl std">(ldaModelInfo)</span>
</pre></div>
<div class="output"><pre class="knitr r"> [1] &quot;label&quot;      &quot;library&quot;    &quot;loop&quot;       &quot;type&quot;       &quot;parameters&quot;
 [6] &quot;grid&quot;       &quot;fit&quot;        &quot;predict&quot;    &quot;prob&quot;       &quot;predictors&quot;
[11] &quot;tags&quot;       &quot;levels&quot;     &quot;sort&quot;      
</pre></div>
</div></div>
<p>
To use your own model, you can pass a list of these components to <tt><span class="hl std">type</span></tt>. This page will describe those components in detail.
</p>

<div id="Illustration1"></div>
<h1>Illustrative Example 1: SVMs with Laplacian Kernels</h1>
<p>
The package currently contains support vector machine (SVM) models using linear, polynomial and radial basis function kernels. The <a href="http://cran.r-project.org/web/packages/kernlab/index.html"><strong>kernlab</strong></a>  package has other functions, including the Laplacian kernel. We will illustrate the model components for this model, which has two parameters: the standard cost parameter for SVMs and one kernel parameter (<tt>sigma</tt>)
</p>


<div id="Components"></div>
<h1>Model Components</h1>


<p> 
You can pass a list of information to the <span class="mx arg">method</span> argument in <span class="mx funCall">train</span>. For models that are built-in to the package, you can just pass the method name as before. 
</p>
<p> 
There are some basic components of the list for custom models. A brief description is below for each then, after setting up and example, each will be described in detail. The list should have the following elements:
</p>
<ul>
  <li>
  <span class="mx arg">library</span> is a character vector of package names that will be needed to fit the model or calculate predictions. <tt><span class="hl kwa">NULL</span></tt> can also be used.
  </li>
  <li>
  <span class="mx arg">type</span> is a simple character vector with values <tt><span class="hl str">&quot;Classification&quot;</span></tt>, <tt><span class="hl str">&quot;Regression&quot;</span></tt> or both. 
  </li>
  <li>
  <span class="mx arg">parameters</span> is a data frame with three simple attributes for each tuning parameter (if any): the argument name (e.g. <span class="mx arg">mtry</span>), the type of data in the parameter grid and textual labels for the parameter.
  </li>
  <li>
  <span class="mx arg">grid</span> is a function that is used to create the tuning grid (unless the user gives the exact values of the parameters via <span class="mx funCall">tuneGrid</span>)
  </li>
  <li>
  <span class="mx arg">fit</span> is a function that fits the model
  </li>
  <li>
  <span class="mx arg">predict</span> is the function that creates predictions
  </li>
  <li>
  <span class="mx arg">prob</span> is a function that can be used to create class probabilities (if applicable)
  </li>
  <li>
  <span class="mx arg">sort</span> is a function that sorts the parameter from most complex to least
  </li>
  <li>
  <span class="mx arg">loop</span> is an <b>optional</b> function for advanced users for models that can create multiple submodel predictions from the same object.
  </li>
  <li>
  <span class="mx arg">levels</span> is an <b>optional</b> function, primarily for classification models using <tt>S4</tt> methods to return the factor levels of the outcome.
  </li>  
  <li>
  <span class="mx arg">tags</span> is an <b>optional</b> character vector that has subjects associated with the model, such as <tt>Tree-Based Model</tt> or <tt>Embedded Feature Selection</tt>. This string is used by the package to create additional documentation pages on the package website. 
  </li>  
  <li>
  <span class="mx arg">label</span> is an <b>optional</b> character string that names the model (e.g. "Linear Discriminant Analysis").
  </li>   
  <li>
  <span class="mx arg">predictors</span> is an <b>optional</b> function that returns a character vector that contains the names of the predictors that we used in the prediction equation.
  </li> 
  <li>
  <span class="mx arg">varImp</span> is an <b>optional</b> function that calculates variable importance metrics for the model (if any).
  </li>    
  <li>
  <span class="mx arg">oob</span> is another <b>optional</b> function that calculates  out-of-bag performance estimates from the model object. Most models do not have this capability but some (e.g. random forests, bagged models) do. 
  </li>    
  <li>
  <span class="mx arg">notes</span> is an <b>optional</b> character vector that can be used to document non-obvious aspects of the model. For example, there are two Bayesian lasso models (<a href="https://github.com/topepo/caret/blob/master/models/files/blasso.R"><tt>blasso</tt></a> and <a href="https://github.com/topepo/caret/blob/master/models/files/blassoAveraged.R"><tt>blassoAveraged</tt></a>) and this field is used to describe the differences between the two models. 
  </li>    
  <li>
  <span class="mx arg">check</span> is an <b>optional</b> function that can be used to check the system/install to make sure that any atypical software requirements are available to the user. The input is <span class="mx arg">pkg</span>, which is the same character string given by the <span class="mx arg">library</span>. This function is run <i>after</i> the checking function to see if the packages specified in <span class="mx arg">library</span> are installed. As an example, the model <a href="https://github.com/topepo/caret/blob/master/models/files/pythonKnnReg.R"><tt>pythonKnnReg</tt></a> uses certain python libraries and the user should have python and these libraries installed. The <a href="https://github.com/topepo/caret/blob/master/models/files/pythonKnnReg.R">model file</a> demonstrates how to check for python libraries prior to running the R model. 
  </li>      
 </ul> 
 
<p>
 In the <a href="http://cran.r-project.org/web/packages/caret/index.html"><strong>caret</strong></a>  package, the subdirectory <tt>models</tt> has all the code for each model that <span class="mx funCall">train</span> interfaces with and these can be used as prototypes for your model. 
</p>
<p>
Let's create a new model for a classification support vector machine using the Laplacian kernel function. We will use the <span class="mx funCall">kernlab</span> package's <span class="mx funCall">ksvm</span> function. The kernel has two parameters: the standard cost parameter for SVMs and one kernel parameter (<tt>sigma</tt>).
</p>
<p>
To start, we'll create a new list:
</p>
<div class="chunk" id="custom_basic"><div class="rcode"><div class="source"><pre class="knitr r">    <span class="hl std">lpSVM</span> <span class="hl kwb">&lt;-</span> <span class="hl kwd">list</span><span class="hl std">(</span><span class="hl kwc">type</span> <span class="hl std">=</span> <span class="hl str">&quot;Classification&quot;</span><span class="hl std">,</span>
                  <span class="hl kwc">library</span> <span class="hl std">=</span> <span class="hl str">&quot;kernlab&quot;</span><span class="hl std">,</span>
                  <span class="hl kwc">loop</span> <span class="hl std">=</span> <span class="hl kwa">NULL</span><span class="hl std">)</span>
</pre></div>
</div></div>
<p>
This model can also be used for regression but we will constrain things here for simplicity. For other SVM models, the type value would be <tt><span class="hl kwd">c</span><span class="hl std">(</span><span class="hl str">&quot;Classification&quot;</span><span class="hl std">,</span> <span class="hl str">&quot;Regression&quot;</span><span class="hl std">)</span></tt>. 
</p>
<p>
The <span class="mx arg">library</span> value checks to see if this package is installed and loads it whenever it is needed (e.g. before modeling or prediction).
</p>

<h2>The parameters Element</h2>

<p>
We have to create some basic information for the parameters in the form of a data frame. The first column is the name of the parameter. The convention is to use the argument name in the model function (e.g. the <span class="mx funCall">ksvm</span> function here). Those values are <tt>C</tt> and <tt>sigma</tt>. Each is a number and we can give them labels of <tt><span class="string">"Cost"</span></tt> and <tt><span class="string">"Sigma"</span></tt>, respectively. The <span class="mx arg">parameters</span> element would then be:
</p>
<div class="chunk" id="custom_labels"><div class="rcode"><div class="source"><pre class="knitr r"><span class="hl std">prm</span> <span class="hl kwb">&lt;-</span> <span class="hl kwd">data.frame</span><span class="hl std">(</span><span class="hl kwc">parameter</span> <span class="hl std">=</span> <span class="hl kwd">c</span><span class="hl std">(</span><span class="hl str">&quot;C&quot;</span><span class="hl std">,</span> <span class="hl str">&quot;sigma&quot;</span><span class="hl std">),</span>
                  <span class="hl kwc">class</span> <span class="hl std">=</span> <span class="hl kwd">rep</span><span class="hl std">(</span><span class="hl str">&quot;numeric&quot;</span><span class="hl std">,</span> <span class="hl num">2</span><span class="hl std">),</span>
                  <span class="hl kwc">label</span> <span class="hl std">=</span> <span class="hl kwd">c</span><span class="hl std">(</span><span class="hl str">&quot;Cost&quot;</span><span class="hl std">,</span> <span class="hl str">&quot;Sigma&quot;</span><span class="hl std">))</span>
</pre></div>
</div></div>
<p>                      
Now we assign it to the model list:
</p>
<div class="chunk" id="unnamed-chunk-1"><div class="rcode"><div class="source"><pre class="knitr r">    <span class="hl std">lpSVM</span><span class="hl opt">$</span><span class="hl std">parameters</span> <span class="hl kwb">&lt;-</span> <span class="hl std">prm</span>
</pre></div>
</div></div>
<p>    
Values of <span class="mx arg">type</span> can indicate numeric, character or logical data types. 
</p>

<h2>The grid Element</h2>
<p> 
This should be a function that takes parameters: <span class="mx arg">x</span> and <span class="mx arg">y</span> (for the predictors and outcome data), <span class="mx arg">len</span> (the number of values per tuning parameter)  as well as <span class="mx arg">search</span>. <span class="mx arg">len</span> is the value of <span class="mx arg">tuneLength</span> that is potentially passed in through <span class="mx funCall">train</span>. <span class="mx arg">search</span> can be either <tt><span class="hl str">&quot;grid&quot;</span></tt> or <tt><span class="hl str">&quot;random&quot;</span></tt>. This can be used to setup a grid for searching or random values for random search. 
</p> 
<p> 
The output should be a data frame of tuning parameter combinations with a column for each parameter. The column names should be the parameter name (e.g. the values of <tt>prm$parameter</tt>). In our case, let's vary the cost parameter on the log 2 scale. For the sigma parameter, we can use the <tt>kernlab</tt> function <span class="mx funCall">sigest</span> to pre-estimate the value. Following <span class="mx funCall">ksvm</span> we take the average of the low and high estimates. Here is a function we could use:
</p> 
<div class="chunk" id="custom_grid"><div class="rcode"><div class="source"><pre class="knitr r"><span class="hl std">svmGrid</span> <span class="hl kwb">&lt;-</span> <span class="hl kwa">function</span><span class="hl std">(</span><span class="hl kwc">x</span><span class="hl std">,</span> <span class="hl kwc">y</span><span class="hl std">,</span> <span class="hl kwc">len</span> <span class="hl std">=</span> <span class="hl kwa">NULL</span><span class="hl std">,</span> <span class="hl kwc">search</span> <span class="hl std">=</span> <span class="hl str">&quot;grid&quot;</span><span class="hl std">) {</span>
  <span class="hl kwd">library</span><span class="hl std">(kernlab)</span>
  <span class="hl com">## This produces low, middle and high values for sigma </span>
  <span class="hl com">## (i.e. a vector with 3 elements). </span>
  <span class="hl std">sigmas</span> <span class="hl kwb">&lt;-</span> <span class="hl kwd">sigest</span><span class="hl std">(</span><span class="hl kwd">as.matrix</span><span class="hl std">(x),</span> <span class="hl kwc">na.action</span> <span class="hl std">= na.omit,</span> <span class="hl kwc">scaled</span> <span class="hl std">=</span> <span class="hl num">TRUE</span><span class="hl std">)</span>
  <span class="hl com">## To use grid search:</span>
  <span class="hl kwa">if</span><span class="hl std">(search</span> <span class="hl opt">==</span> <span class="hl str">&quot;grid&quot;</span><span class="hl std">) {</span>
    <span class="hl std">out</span> <span class="hl kwb">&lt;-</span> <span class="hl kwd">expand.grid</span><span class="hl std">(</span><span class="hl kwc">sigma</span> <span class="hl std">=</span> <span class="hl kwd">mean</span><span class="hl std">(</span><span class="hl kwd">as.vector</span><span class="hl std">(sigmas[</span><span class="hl opt">-</span><span class="hl num">2</span><span class="hl std">])),</span>
                       <span class="hl kwc">C</span> <span class="hl std">=</span> <span class="hl num">2</span> <span class="hl opt">^</span><span class="hl std">((</span><span class="hl num">1</span><span class="hl opt">:</span><span class="hl std">len)</span> <span class="hl opt">-</span> <span class="hl num">3</span><span class="hl std">))</span>
  <span class="hl std">}</span> <span class="hl kwa">else</span> <span class="hl std">{</span>
    <span class="hl com">## For random search, define ranges for the parameters then</span>
    <span class="hl com">## generate random values for them</span>
    <span class="hl std">rng</span> <span class="hl kwb">&lt;-</span> <span class="hl kwd">extendrange</span><span class="hl std">(</span><span class="hl kwd">log</span><span class="hl std">(sigmas),</span> <span class="hl kwc">f</span> <span class="hl std">=</span> <span class="hl num">.75</span><span class="hl std">)</span>
    <span class="hl std">out</span> <span class="hl kwb">&lt;-</span> <span class="hl kwd">data.frame</span><span class="hl std">(</span><span class="hl kwc">sigma</span> <span class="hl std">=</span> <span class="hl kwd">exp</span><span class="hl std">(</span><span class="hl kwd">runif</span><span class="hl std">(len,</span> <span class="hl kwc">min</span> <span class="hl std">= rng[</span><span class="hl num">1</span><span class="hl std">],</span> <span class="hl kwc">max</span> <span class="hl std">= rng[</span><span class="hl num">2</span><span class="hl std">])),</span>
                      <span class="hl kwc">C</span> <span class="hl std">=</span> <span class="hl num">2</span><span class="hl opt">^</span><span class="hl kwd">runif</span><span class="hl std">(len,</span> <span class="hl kwc">min</span> <span class="hl std">=</span> <span class="hl opt">-</span><span class="hl num">5</span><span class="hl std">,</span> <span class="hl kwc">max</span> <span class="hl std">=</span> <span class="hl num">8</span><span class="hl std">))</span>
  <span class="hl std">}</span>
  <span class="hl std">out</span>
<span class="hl std">}</span>
</pre></div>
</div></div>
<p> 
Again, the user can pass their own grid via <span class="mx funCall">train</span>'s <span class="mx arg">tuneGrid</span> option or they can use this code to create a default grid. 
</p> 
<p> 
We assign this function to the overall model list:
</p> 
<div class="chunk" id="custom_grid2"><div class="rcode"><div class="source"><pre class="knitr r">    <span class="hl std">lpSVM</span><span class="hl opt">$</span><span class="hl std">grid</span> <span class="hl kwb">&lt;-</span> <span class="hl std">svmGrid</span>
</pre></div>
</div></div>

<h2>The fit Element</h2>

<p>
Here is where we fit the model. This <span class="mx funCall">fit</span> function has several arguments:
<ul>
  <li>
  <span class="mx arg">x</span>, <span class="mx arg">y</span>: the current data used to fit the model
  </li>
  <li>
  <span class="mx arg">wts</span>: optional instance weights (not applicable for this particular model)
  </li>
  <li>
  <span class="mx arg">param</span>: the current tuning parameter values
  </li>
  <li>
  <span class="mx arg">lev</span>: the class levels of the outcome (or <tt><span class="hl kwa">NULL</span></tt> in regression)
  </li>
  <li>
  <span class="mx arg">last</span>: a logical for whether the current fit is the final fit
  </li>
  <li>
  <span class="mx arg">weights</span>
  </li>
  <li>
  <span class="mx arg">classProbs</span>: a logical for whether class probabilities should be computed.
  </li>
</ul>
<p>
Here is something we could use for this model:
<p>
<div class="chunk" id="custom_fit"><div class="rcode"><div class="source"><pre class="knitr r"><span class="hl std">svmFit</span> <span class="hl kwb">&lt;-</span> <span class="hl kwa">function</span><span class="hl std">(</span><span class="hl kwc">x</span><span class="hl std">,</span> <span class="hl kwc">y</span><span class="hl std">,</span> <span class="hl kwc">wts</span><span class="hl std">,</span> <span class="hl kwc">param</span><span class="hl std">,</span> <span class="hl kwc">lev</span><span class="hl std">,</span> <span class="hl kwc">last</span><span class="hl std">,</span> <span class="hl kwc">weights</span><span class="hl std">,</span> <span class="hl kwc">classProbs</span><span class="hl std">,</span> <span class="hl kwc">...</span><span class="hl std">) {</span>
  <span class="hl kwd">ksvm</span><span class="hl std">(</span><span class="hl kwc">x</span> <span class="hl std">=</span> <span class="hl kwd">as.matrix</span><span class="hl std">(x),</span> <span class="hl kwc">y</span> <span class="hl std">= y,</span>
       <span class="hl kwc">kernel</span> <span class="hl std">= rbfdot,</span>
       <span class="hl kwc">kpar</span> <span class="hl std">=</span> <span class="hl kwd">list</span><span class="hl std">(</span><span class="hl kwc">sigma</span> <span class="hl std">= param</span><span class="hl opt">$</span><span class="hl std">sigma),</span>
       <span class="hl kwc">C</span> <span class="hl std">= param</span><span class="hl opt">$</span><span class="hl std">C,</span>
       <span class="hl kwc">prob.model</span> <span class="hl std">= classProbs,</span>
       <span class="hl std">...)</span>
 <span class="hl std">}</span>

<span class="hl std">lpSVM</span><span class="hl opt">$</span><span class="hl std">fit</span> <span class="hl kwb">&lt;-</span> <span class="hl std">svmFit</span>
</pre></div>
</div></div>
<p>
A few notes about this:
</p>
<ul>
  <li>
  Notice that the package is not loaded in the code. It is loaded prior to this function being called so it won't hurt if you load it again (but that's not needed).
  </li> 
  <li>
  The <span class="mx funCall">ksvm</span> function requires a <i>matrix</i> or predictors. If the original data were a data frame, this would throw and error. 
  </li> 
  <li>
  The tuning parameters are references in the <span class="mx arg">param</span> data frame. There is always a single row in this data frame. 
  </li> 
  <li>
  The probability model is fit based on the value of <span class="mx arg">classProbs</span>. This value is determined by the value given in <span class="mx funCall">trainControl</span>. 
  </li> 
  <li>
  The three dots allow the user to pass options in from <span class="mx funCall">train</span> to, in this case, the <span class="mx funCall">ksvm</span> function. For example, if the user wanted to set the cache size for the function, they could list <span class="mx arg">cache</span><tt> = <span class="hl num">80</span></tt> and this argument will be pass from <span class="mx funCall">train</span> to <span class="mx funCall">ksvm</span>.  
  </li> 
  <li> 
  Any pre-processing that was requested in the call to <span class="mx funCall">train</span> have been done. For example, if <span class="mx arg">preProc</span><tt> = <span class="hl str">&quot;center&quot;</span></tt> was originally requested, the columns of <tt>x</tt> seen within this function are mean centered. 
  </li> 
</ul>

<h2>The predict Element</h2>

<p>
This is a function that produces a vector or predictions. In our case, these are class predictions but they could be numbers for regression models. 
</p>
<p>
The arguments are:
</p>
<ul>
  <li>
  <span class="mx arg">modelFit</span>: the model produced by the <tt>fit</tt> code shown above. 
  </li>
  <li>
  <span class="mx arg">newdata</span>: the predictor values of the instances being predicted (e.g. out-of-bag samples)
  </li>
  <li>
  <span class="mx arg">preProc</span> 
  </li>
  <li>
  <span class="mx arg">submodels</span>: this an optional list of tuning parameters only used with the <span class="mx arg">loop</span> element discussed below. In most cases, it will be <tt><span class="hl kwa">NULL</span></tt>.
  </li>
</ul>
<p>
Our function will be very simple:
</p>
<div class="chunk" id="custom_pred"><div class="rcode"><div class="source"><pre class="knitr r"><span class="hl std">svmPred</span> <span class="hl kwb">&lt;-</span> <span class="hl kwa">function</span><span class="hl std">(</span><span class="hl kwc">modelFit</span><span class="hl std">,</span> <span class="hl kwc">newdata</span><span class="hl std">,</span> <span class="hl kwc">preProc</span> <span class="hl std">=</span> <span class="hl kwa">NULL</span><span class="hl std">,</span> <span class="hl kwc">submodels</span> <span class="hl std">=</span> <span class="hl kwa">NULL</span><span class="hl std">)</span>
   <span class="hl kwd">predict</span><span class="hl std">(modelFit, newdata)</span>
<span class="hl std">lpSVM</span><span class="hl opt">$</span><span class="hl std">predict</span> <span class="hl kwb">&lt;-</span> <span class="hl std">svmPred</span>
</pre></div>
</div></div>
<p></p>

<p>    
The function <span class="mx funCall">predict.ksvm</span> will automatically create a factor vector as output. The function could also produce character values. Either way, the innards of <span class="mx funCall">train</span> will make them factors and ensure that the same levels as the original data are used. 
</p>

<h2>The prob Element</h2>

<p>
If a regression model is being used or if the classification model does not create class probabilities a value of <tt><span class="hl kwa">NULL</span></tt> can be used here instead of a function. Otherwise, the function arguments are the same as the <span class="mx arg">pred</span> function. The output should be a matrix or data frame of class probabilities with a column for each class. The column names should be the class levels. 
</p>
<p>
We can use:
</p>

<div class="chunk" id="custom_prob"><div class="rcode"><div class="source"><pre class="knitr r"><span class="hl std">svmProb</span> <span class="hl kwb">&lt;-</span> <span class="hl kwa">function</span><span class="hl std">(</span><span class="hl kwc">modelFit</span><span class="hl std">,</span> <span class="hl kwc">newdata</span><span class="hl std">,</span> <span class="hl kwc">preProc</span> <span class="hl std">=</span> <span class="hl kwa">NULL</span><span class="hl std">,</span> <span class="hl kwc">submodels</span> <span class="hl std">=</span> <span class="hl kwa">NULL</span><span class="hl std">)</span>
   <span class="hl kwd">predict</span><span class="hl std">(modelFit, newdata,</span> <span class="hl kwc">type</span><span class="hl std">=</span><span class="hl str">&quot;probabilities&quot;</span><span class="hl std">)</span>
<span class="hl std">lpSVM</span><span class="hl opt">$</span><span class="hl std">prob</span> <span class="hl kwb">&lt;-</span> <span class="hl std">svmProb</span>
</pre></div>
</div></div>

<p></p>
<p>
If you look at some of the SVM examples in the <tt>models</tt> directory, the real functions used by <span class="mx funCall">train</span> are much more complicated so that they can deal with model failures, probabilities that do not sum to 1 etc.
</p>

<h2>The sort Element</h2>

<p>
This is an optional function that sorts the tuning parameters from the simplest model to the most complex. There are times where this ordering is not obvious. This information is used when the performance values are tied across multiple parameters. We would probably want to choose the least complex model in those cases. 
</p>

<p>
Here, we will sort by the cost value. Smaller values of <tt>C</tt> produce smoother class boundaries than larger values:
</p>

<div class="chunk" id="custom_sort"><div class="rcode"><div class="source"><pre class="knitr r">    <span class="hl std">svmSort</span> <span class="hl kwb">&lt;-</span> <span class="hl kwa">function</span><span class="hl std">(</span><span class="hl kwc">x</span><span class="hl std">) x[</span><span class="hl kwd">order</span><span class="hl std">(x</span><span class="hl opt">$</span><span class="hl std">C),]</span>
    <span class="hl std">lpSVM</span><span class="hl opt">$</span><span class="hl std">sort</span> <span class="hl kwb">&lt;-</span> <span class="hl std">svmSort</span>
</pre></div>
</div></div>


<h2>The levels Element</h2>

<p>
<span class="mx funCall">train</span> ensures that classification models always predict factors with the same levels. To do this at prediction time, the package needs to know the levels from the model object (specifically, the <tt><span class="hl std">finalModels</span></tt> slot of the <span class="mx funCall">train</span> object).
</p>
<p>
For model functions using <code>S3</code> methods, <span class="mx funCall">train</span> automatically attaches a character vector called <tt><span class="hl std">obsLevels</span></tt> to the object and the package code uses this value. However, this strategy does not work for <code>S4</code> methods. In these cases, the package will use the code found in the <tt><span class="hl std">levels</span></tt> slot of the model list. 
</p>
<p>
For example, the <span class="mx funCall">ksvm</span> function uses <code>S4</code> methods but, unlike most model functions, has a built--in function called <span class="mx funCall">lev</span> that will extract the class levels (if any). In this case, our levels code would be:
</p>
<div class="chunk" id="custom_lev"><div class="rcode"><div class="source"><pre class="knitr r">    <span class="hl std">lpSVM</span><span class="hl opt">$</span><span class="hl std">levels</span> <span class="hl kwb">&lt;-</span> <span class="hl kwa">function</span><span class="hl std">(</span><span class="hl kwc">x</span><span class="hl std">)</span> <span class="hl kwd">lev</span><span class="hl std">(x)</span>
</pre></div>
</div></div>
<p>
In most other cases, the levels will beed to be extracted from data contained in the fitted model object. As another example, objects created using the <span class="mx funCall">ctree</span> function in the <span class="mx funCall">party</span> package would need to use:
</p>

<div class="chunk" id="custom_lev2"><div class="rcode"><div class="source"><pre class="knitr r">    <span class="hl kwa">function</span><span class="hl std">(</span><span class="hl kwc">x</span><span class="hl std">)</span> <span class="hl kwd">levels</span><span class="hl std">(x</span><span class="hl opt">@</span><span class="hl kwc">data</span><span class="hl opt">@</span><span class="hl kwc">get</span><span class="hl std">(</span><span class="hl str">&quot;response&quot;</span><span class="hl std">)[,</span><span class="hl num">1</span><span class="hl std">])</span>
</pre></div>
</div></div>
<p>
Again, this slot is only used for classification models using <code>S4</code> methods.
</p>

<p>
We should now be ready to fit our model. 
</p>
<div class="chunk" id="custom_sonar"><div class="rcode"><div class="source"><pre class="knitr r"><span class="hl kwd">library</span><span class="hl std">(mlbench)</span>
</pre></div>
<div class="error"><pre class="knitr r">Error in library(mlbench): there is no package called 'mlbench'
</pre></div>
<div class="source"><pre class="knitr r"><span class="hl kwd">data</span><span class="hl std">(Sonar)</span>

<span class="hl kwd">library</span><span class="hl std">(caret)</span>
<span class="hl kwd">set.seed</span><span class="hl std">(</span><span class="hl num">998</span><span class="hl std">)</span>
<span class="hl std">inTraining</span> <span class="hl kwb">&lt;-</span> <span class="hl kwd">createDataPartition</span><span class="hl std">(Sonar</span><span class="hl opt">$</span><span class="hl std">Class,</span> <span class="hl kwc">p</span> <span class="hl std">=</span> <span class="hl num">.75</span><span class="hl std">,</span> <span class="hl kwc">list</span> <span class="hl std">=</span> <span class="hl num">FALSE</span><span class="hl std">)</span>
</pre></div>
<div class="error"><pre class="knitr r">Error in createDataPartition(Sonar$Class, p = 0.75, list = FALSE): object 'Sonar' not found
</pre></div>
<div class="source"><pre class="knitr r"><span class="hl std">training</span> <span class="hl kwb">&lt;-</span> <span class="hl std">Sonar[ inTraining,]</span>
</pre></div>
<div class="error"><pre class="knitr r">Error in eval(expr, envir, enclos): object 'Sonar' not found
</pre></div>
<div class="source"><pre class="knitr r"><span class="hl std">testing</span>  <span class="hl kwb">&lt;-</span> <span class="hl std">Sonar[</span><span class="hl opt">-</span><span class="hl std">inTraining,]</span>
</pre></div>
<div class="error"><pre class="knitr r">Error in eval(expr, envir, enclos): object 'Sonar' not found
</pre></div>
<div class="source"><pre class="knitr r"><span class="hl std">fitControl</span> <span class="hl kwb">&lt;-</span> <span class="hl kwd">trainControl</span><span class="hl std">(</span><span class="hl kwc">method</span> <span class="hl std">=</span> <span class="hl str">&quot;repeatedcv&quot;</span><span class="hl std">,</span>
                           <span class="hl com">## 10-fold CV...</span>
                           <span class="hl kwc">number</span> <span class="hl std">=</span> <span class="hl num">10</span><span class="hl std">,</span>
                           <span class="hl com">## repeated ten times</span>
                           <span class="hl kwc">repeats</span> <span class="hl std">=</span> <span class="hl num">10</span><span class="hl std">)</span>

<span class="hl kwd">set.seed</span><span class="hl std">(</span><span class="hl num">825</span><span class="hl std">)</span>
<span class="hl std">Laplacian</span> <span class="hl kwb">&lt;-</span> <span class="hl kwd">train</span><span class="hl std">(Class</span> <span class="hl opt">~</span> <span class="hl std">.,</span> <span class="hl kwc">data</span> <span class="hl std">= training,</span>
                   <span class="hl kwc">method</span> <span class="hl std">= lpSVM,</span>
                   <span class="hl kwc">preProc</span> <span class="hl std">=</span> <span class="hl kwd">c</span><span class="hl std">(</span><span class="hl str">&quot;center&quot;</span><span class="hl std">,</span> <span class="hl str">&quot;scale&quot;</span><span class="hl std">),</span>
                   <span class="hl kwc">tuneLength</span> <span class="hl std">=</span> <span class="hl num">8</span><span class="hl std">,</span>
                   <span class="hl kwc">trControl</span> <span class="hl std">= fitControl)</span>
</pre></div>
<div class="error"><pre class="knitr r">Error in eval(expr, envir, enclos): object 'training' not found
</pre></div>
<div class="source"><pre class="knitr r"><span class="hl kwd">print</span><span class="hl std">(Laplacian,</span> <span class="hl kwc">digits</span> <span class="hl std">=</span> <span class="hl num">3</span><span class="hl std">)</span>
</pre></div>
<div class="error"><pre class="knitr r">Error in print(Laplacian, digits = 3): object 'Laplacian' not found
</pre></div>
</div></div>

<p>
A plot of the data shows that the model doesn't change when the cost value is above 16.
</p>
<div class="chunk" id="custom_trainplot, Laplacian"><div class="rcode"><div class="source"><pre class="knitr r"><span class="hl kwd">trellis.par.set</span><span class="hl std">(</span><span class="hl kwd">caretTheme</span><span class="hl std">())</span>
<span class="hl kwd">plot</span><span class="hl std">(Laplacian,</span>  <span class="hl kwc">scales</span> <span class="hl std">=</span> <span class="hl kwd">list</span><span class="hl std">(</span><span class="hl kwc">x</span> <span class="hl std">=</span> <span class="hl kwd">list</span><span class="hl std">(</span><span class="hl kwc">log</span> <span class="hl std">=</span> <span class="hl num">2</span><span class="hl std">)))</span>
</pre></div>
<div class="error"><pre class="knitr r">Error in plot(Laplacian, scales = list(x = list(log = 2))): object 'Laplacian' not found
</pre></div>
</div></div>

<div id="Illustration2"></div>
<h1>Illustrative Example 2: Something More Complicated - LogitBoost</h1>

<h2>The loop Element</h2>

<p>
This function can be used to create custom loops for models to tune over. In most cases, the function can just return the existing tuning grid.
</p>
<p>
For example, a <span class="mx funCall">LogitBoost</span> model can be trained over the number of boosting iterations. In the <a href="http://cran.r-project.org/web/packages/caTools/index.html"><strong>caTools</strong></a> package, the <span class="mx funCall">LogitBoost</span> function can be used to fit this model. For example:
</p>
<div class="chunk" id="custom_lb"><div class="rcode"><div class="source"><pre class="knitr r"><span class="hl std">mod</span> <span class="hl kwb">&lt;-</span> <span class="hl kwd">LogitBoost</span><span class="hl std">(</span><span class="hl kwd">as.matrix</span><span class="hl std">(x), y,</span> <span class="hl kwc">nIter</span> <span class="hl std">=</span> <span class="hl num">51</span><span class="hl std">)</span>
</pre></div>
</div></div>

<p>
If we were to tune the model evaluating models where the number of iterations was 11, 21, 31, 41 and 51, the grid could be
</p>
<div class="chunk" id="custom_lb_grid"><div class="rcode"><div class="source"><pre class="knitr r"><span class="hl std">lbGrid</span> <span class="hl kwb">&lt;-</span> <span class="hl kwd">data.frame</span><span class="hl std">(</span><span class="hl kwc">nIter</span> <span class="hl std">=</span> <span class="hl kwd">seq</span><span class="hl std">(</span><span class="hl num">11</span><span class="hl std">,</span> <span class="hl num">51</span><span class="hl std">,</span> <span class="hl kwc">by</span> <span class="hl std">=</span> <span class="hl num">10</span><span class="hl std">))</span>
</pre></div>
</div></div>

<p>
During resampling, <span class="mx funCall">train</span> could loop over all five rows in <code>lbGrid</code> and fit five models. However, the <span class="mx funCall">predict.LogitBoost</span> function has an argument called <span class="mx arg">nIter</span> that can produce, in this case, predictions from <code>mod</code> for all five models. 
</p>
<p>
Instead of <span class="mx funCall">train</span> fitting five models, we could fit a single model with <span class="mx arg">nIter</span> = <span class="hl num">51</span> and derive predictions for all five models using only <code>mod</code>. 
</p>
<p>
The terminology used here is that  <span class="mx arg">nIter</span> is a <i>sequential</i> tuning parameter (and the other parameters would be considered <i>fixed</i>). 
</p>
<p>
The <span class="mx arg">loop</span> argument for models is used to produce two objects:
</p>
<ul>
 <li> <code>loop</code>: this is the actual loop that is used by `train`. 
 </li>
 <li>
 <code>submodels</code> is a <i>list</i> that has as many elements as there are rows in <code>loop</code>. The list has all the "extra" parameter settings that can be derived for each model.
 </li>
</ul>
<p>
Going back to the <span class="mx funCall">LogitBoost</span> example, we could have:
</p>
<div class="chunk" id="custom_lb_loops"><div class="rcode"><div class="source"><pre class="knitr r">    <span class="hl std">loop</span> <span class="hl kwb">&lt;-</span> <span class="hl kwd">data.frame</span><span class="hl std">(</span><span class="hl kwc">.nIter</span> <span class="hl std">=</span> <span class="hl num">51</span><span class="hl std">)</span>
<span class="hl std">loop</span>
</pre></div>
<div class="output"><pre class="knitr r">  .nIter
1     51
</pre></div>
<div class="source"><pre class="knitr r">    <span class="hl std">submodels</span> <span class="hl kwb">&lt;-</span> <span class="hl kwd">list</span><span class="hl std">(</span><span class="hl kwd">data.frame</span><span class="hl std">(</span><span class="hl kwc">nIter</span> <span class="hl std">=</span> <span class="hl kwd">seq</span><span class="hl std">(</span><span class="hl num">11</span><span class="hl std">,</span> <span class="hl num">41</span><span class="hl std">,</span> <span class="hl kwc">by</span> <span class="hl std">=</span> <span class="hl num">10</span><span class="hl std">)))</span>
<span class="hl std">submodels</span>
</pre></div>
<div class="output"><pre class="knitr r">[[1]]
  nIter
1    11
2    21
3    31
4    41
</pre></div>
</div></div>
<p>
For this case, <span class="mx funCall">train</span> first fits the <span class="mx arg">nIter</span><tt> = <span class="hl num">51</span></tt> model. When the model is predicted, that code has a <span class="mx funCall">for</span> loop that iterates over the elements of <code>submodel[[1]]</code> to get the predictions for the other 4 models. 
</p>
<p>
In the end, predictions for all five models (for <span class="mx arg">nIter</span><tt> = <span class="hl kwd">seq</span><span class="hl std">(</span><span class="hl num">11</span><span class="hl std">,</span> <span class="hl num">51</span><span class="hl std">,</span> <span class="hl kwc">by</span> <span class="hl std">=</span> <span class="hl num">10</span><span class="hl std">)</span></tt>) with a single model fit. 
</p>
<p>
There are other models built-in to <a href="http://cran.r-project.org/web/packages/caret/index.html"><strong>caret</strong></a> that are used this way. There are a number of models that have multiple sequential tuning parameters.
</p>
<p>
If the <span class="mx arg">loop</span> argument is left <code><span class="hl kwa">NULL</span></code> the results of <span class="mx arg">tuneGrid</span> are used as the simple loop and is recommended for most situations. Note that the machinery that is used to "derive" the extra predictions is up to the user to create, typically in the <span class="mx arg">predict</span> and <span class="mx arg">prob</span>  elements of the custom model object. 
</p>
<p>    
For the <span class="mx funCall">LogitBoost</span> model, some simple code to create these objects would be:
</p>

<div class="chunk" id="custom_lb_loop_funcs"><div class="rcode"><div class="source"><pre class="knitr r">    <span class="hl std">fullGrid</span> <span class="hl kwb">&lt;-</span> <span class="hl kwd">data.frame</span><span class="hl std">(</span><span class="hl kwc">nIter</span> <span class="hl std">=</span> <span class="hl kwd">seq</span><span class="hl std">(</span><span class="hl num">11</span><span class="hl std">,</span> <span class="hl num">51</span><span class="hl std">,</span> <span class="hl kwc">by</span> <span class="hl std">=</span> <span class="hl num">10</span><span class="hl std">))</span>

    <span class="hl com">## Get the largest value of nIter to fit the &quot;full&quot; model</span>
    <span class="hl std">loop</span> <span class="hl kwb">&lt;-</span> <span class="hl std">fullGrid[</span><span class="hl kwd">which.max</span><span class="hl std">(fullGrid</span><span class="hl opt">$</span><span class="hl std">nIter),,</span><span class="hl kwc">drop</span> <span class="hl std">=</span> <span class="hl num">FALSE</span><span class="hl std">]</span>
    <span class="hl std">loop</span>
</pre></div>
<div class="output"><pre class="knitr r">  nIter
5    51
</pre></div>
<div class="source"><pre class="knitr r">    <span class="hl std">submodels</span> <span class="hl kwb">&lt;-</span> <span class="hl std">fullGrid[</span><span class="hl opt">-</span><span class="hl kwd">which.max</span><span class="hl std">(fullGrid</span><span class="hl opt">$</span><span class="hl std">nIter),,</span><span class="hl kwc">drop</span> <span class="hl std">=</span> <span class="hl num">FALSE</span><span class="hl std">]</span>

    <span class="hl com">## This needs to be encased in a list in case there are more</span>
    <span class="hl com">## than one tuning parameter</span>
    <span class="hl std">submodels</span> <span class="hl kwb">&lt;-</span> <span class="hl kwd">list</span><span class="hl std">(submodels)</span>
    <span class="hl std">submodels</span>
</pre></div>
<div class="output"><pre class="knitr r">[[1]]
  nIter
1    11
2    21
3    31
4    41
</pre></div>
</div></div>
<p>
For the <span class="mx funCall">LogitBoost</span> custom model object, we could use this code in the <span class="mx arg">predict</span> slot:
<p>
<div class="chunk" id="custom_lb_pred"><div class="rcode"><div class="source"><pre class="knitr r"><span class="hl std">lbPred</span> <span class="hl kwb">&lt;-</span> <span class="hl kwa">function</span><span class="hl std">(</span><span class="hl kwc">modelFit</span><span class="hl std">,</span> <span class="hl kwc">newdata</span><span class="hl std">,</span> <span class="hl kwc">preProc</span> <span class="hl std">=</span> <span class="hl kwa">NULL</span><span class="hl std">,</span> <span class="hl kwc">submodels</span> <span class="hl std">=</span> <span class="hl kwa">NULL</span><span class="hl std">) {</span>
  <span class="hl com">## This model was fit with the maximum value of nIter</span>
  <span class="hl std">out</span> <span class="hl kwb">&lt;-</span> <span class="hl std">caTools</span><span class="hl opt">::</span><span class="hl kwd">predict.LogitBoost</span><span class="hl std">(modelFit, newdata,</span> <span class="hl kwc">type</span><span class="hl std">=</span><span class="hl str">&quot;class&quot;</span><span class="hl std">)</span>

  <span class="hl com">## In this case, 'submodels' is a data frame with the other values of</span>
  <span class="hl com">## nIter. We loop over these to get the other predictions.</span>
  <span class="hl kwa">if</span><span class="hl std">(</span><span class="hl opt">!</span><span class="hl kwd">is.null</span><span class="hl std">(submodels)) {</span>
    <span class="hl com">## Save _all_ the predictions in a list</span>
    <span class="hl std">tmp</span> <span class="hl kwb">&lt;-</span> <span class="hl std">out</span>
    <span class="hl std">out</span> <span class="hl kwb">&lt;-</span> <span class="hl kwd">vector</span><span class="hl std">(</span><span class="hl kwc">mode</span> <span class="hl std">=</span> <span class="hl str">&quot;list&quot;</span><span class="hl std">,</span> <span class="hl kwc">length</span> <span class="hl std">=</span> <span class="hl kwd">nrow</span><span class="hl std">(submodels)</span> <span class="hl opt">+</span> <span class="hl num">1</span><span class="hl std">)</span>
    <span class="hl std">out[[</span><span class="hl num">1</span><span class="hl std">]]</span> <span class="hl kwb">&lt;-</span> <span class="hl std">tmp</span>

    <span class="hl kwa">for</span><span class="hl std">(j</span> <span class="hl kwa">in</span> <span class="hl kwd">seq</span><span class="hl std">(</span><span class="hl kwc">along</span> <span class="hl std">= submodels</span><span class="hl opt">$</span><span class="hl std">nIter)) {</span>
      <span class="hl std">out[[j</span><span class="hl opt">+</span><span class="hl num">1</span><span class="hl std">]]</span> <span class="hl kwb">&lt;-</span> <span class="hl std">caTools</span><span class="hl opt">::</span><span class="hl kwd">predict.LogitBoost</span><span class="hl std">(modelFit,</span>
                                                <span class="hl std">newdata,</span>
                                                <span class="hl kwc">nIter</span> <span class="hl std">= submodels</span><span class="hl opt">$</span><span class="hl std">nIter[j])</span>
      <span class="hl std">}</span>
    <span class="hl std">}</span>
  <span class="hl std">out</span>
  <span class="hl std">}</span>
</pre></div>
</div></div>

<p>
A few more notes:
</p>
<ul>
 <li> 
 The code in the <span class="mx arg">fit</span> element does not have to change. 
 </li>
 <li>
 The <span class="mx arg">prob</span> slot works in the same way. The only difference is that the values saved in the outgoing lists are matrices or data frames of probabilities for each class. 
  </li>
  <li>
  After model training (i.e. predicting new samples), the value of <span class="mx arg">submodels</span> is set to <code><span class="hl kwa">NULL</span></code> and the code produces a single set of predictions.
  </li>
  <li>
  If the model had one sequential parameter and one fixed parameter, the <code>loop</code> data frame would have two columns (one for each parameter). If the model is tuned over more than one value of the fixed parameter, the <code>submodels</code> list would have more than one element. If <code>loop</code> had 10 rows, then <tt><span class="hl kwd">length</span><span class="hl std">(submodels)</span></tt> would be <tt>10</tt> and <tt><span class="hl std">loop[i,]</span></tt> would be linked to <tt><span class="hl std">submodels[[i]]</span></tt>. 
  </li>
</ul>

<p> Here is a slimmed down version of the logitBoost code already in the package:
</p>

<div class="chunk" id="custom_lb_funcs"><div class="rcode"><div class="source"><pre class="knitr r"><span class="hl std">lbFuncs</span> <span class="hl kwb">&lt;-</span> <span class="hl kwd">list</span><span class="hl std">(</span><span class="hl kwc">library</span> <span class="hl std">=</span> <span class="hl str">&quot;caTools&quot;</span><span class="hl std">,</span>
                <span class="hl kwc">loop</span> <span class="hl std">=</span> <span class="hl kwa">function</span><span class="hl std">(</span><span class="hl kwc">grid</span><span class="hl std">) {</span>
                  <span class="hl std">loop</span> <span class="hl kwb">&lt;-</span> <span class="hl std">grid[</span><span class="hl kwd">which.max</span><span class="hl std">(grid</span><span class="hl opt">$</span><span class="hl std">nIter),,</span><span class="hl kwc">drop</span> <span class="hl std">=</span> <span class="hl num">FALSE</span><span class="hl std">]</span>
                  <span class="hl std">submodels</span> <span class="hl kwb">&lt;-</span> <span class="hl std">grid[</span><span class="hl opt">-</span><span class="hl kwd">which.max</span><span class="hl std">(grid</span><span class="hl opt">$</span><span class="hl std">nIter),,</span><span class="hl kwc">drop</span> <span class="hl std">=</span> <span class="hl num">FALSE</span><span class="hl std">]</span>
                  <span class="hl std">submodels</span> <span class="hl kwb">&lt;-</span> <span class="hl kwd">list</span><span class="hl std">(submodels)</span>
                  <span class="hl kwd">list</span><span class="hl std">(</span><span class="hl kwc">loop</span> <span class="hl std">= loop,</span> <span class="hl kwc">submodels</span> <span class="hl std">= submodels)</span>
                <span class="hl std">},</span>
                <span class="hl kwc">type</span> <span class="hl std">=</span> <span class="hl str">&quot;Classification&quot;</span><span class="hl std">,</span>
                <span class="hl kwc">parameters</span> <span class="hl std">=</span> <span class="hl kwd">data.frame</span><span class="hl std">(</span><span class="hl kwc">parameter</span> <span class="hl std">=</span> <span class="hl str">'nIter'</span><span class="hl std">,</span>
                                        <span class="hl kwc">class</span> <span class="hl std">=</span> <span class="hl str">'numeric'</span><span class="hl std">,</span>
                                        <span class="hl kwc">label</span> <span class="hl std">=</span> <span class="hl str">'# Boosting Iterations'</span><span class="hl std">),</span>
                <span class="hl kwc">grid</span> <span class="hl std">=</span> <span class="hl kwa">function</span><span class="hl std">(</span><span class="hl kwc">x</span><span class="hl std">,</span> <span class="hl kwc">y</span><span class="hl std">,</span> <span class="hl kwc">len</span> <span class="hl std">=</span> <span class="hl kwa">NULL</span><span class="hl std">,</span> <span class="hl kwc">search</span> <span class="hl std">=</span> <span class="hl str">&quot;grid&quot;</span><span class="hl std">) {</span>
                  <span class="hl std">out</span> <span class="hl kwb">&lt;-</span> <span class="hl kwa">if</span><span class="hl std">(search</span> <span class="hl opt">==</span> <span class="hl str">&quot;grid&quot;</span><span class="hl std">)</span>
                    <span class="hl kwd">data.frame</span><span class="hl std">(</span><span class="hl kwc">nIter</span> <span class="hl std">=</span> <span class="hl num">1</span> <span class="hl opt">+</span> <span class="hl std">((</span><span class="hl num">1</span><span class="hl opt">:</span><span class="hl std">len)</span><span class="hl opt">*</span><span class="hl num">10</span><span class="hl std">))</span> <span class="hl kwa">else</span>
                    <span class="hl kwd">data.frame</span><span class="hl std">(</span><span class="hl kwc">nIter</span> <span class="hl std">=</span> <span class="hl kwd">sample</span><span class="hl std">(</span><span class="hl num">1</span><span class="hl opt">:</span><span class="hl num">500</span><span class="hl std">,</span> <span class="hl kwc">size</span> <span class="hl std">= len))</span>
                  <span class="hl std">out</span>
                <span class="hl std">},</span>
                <span class="hl kwc">fit</span> <span class="hl std">=</span> <span class="hl kwa">function</span><span class="hl std">(</span><span class="hl kwc">x</span><span class="hl std">,</span> <span class="hl kwc">y</span><span class="hl std">,</span> <span class="hl kwc">wts</span><span class="hl std">,</span> <span class="hl kwc">param</span><span class="hl std">,</span> <span class="hl kwc">lev</span><span class="hl std">,</span> <span class="hl kwc">last</span><span class="hl std">,</span> <span class="hl kwc">weights</span><span class="hl std">,</span> <span class="hl kwc">classProbs</span><span class="hl std">,</span> <span class="hl kwc">...</span><span class="hl std">) {</span>
                  <span class="hl kwd">LogitBoost</span><span class="hl std">(</span><span class="hl kwd">as.matrix</span><span class="hl std">(x), y,</span> <span class="hl kwc">nIter</span> <span class="hl std">= param</span><span class="hl opt">$</span><span class="hl std">nIter)</span>
                <span class="hl std">},</span>
                <span class="hl kwc">predict</span> <span class="hl std">=</span> <span class="hl kwa">function</span><span class="hl std">(</span><span class="hl kwc">modelFit</span><span class="hl std">,</span> <span class="hl kwc">newdata</span><span class="hl std">,</span> <span class="hl kwc">preProc</span> <span class="hl std">=</span> <span class="hl kwa">NULL</span><span class="hl std">,</span> <span class="hl kwc">submodels</span> <span class="hl std">=</span> <span class="hl kwa">NULL</span><span class="hl std">) {</span>
                  <span class="hl std">out</span> <span class="hl kwb">&lt;-</span> <span class="hl kwd">predict</span><span class="hl std">(modelFit, newdata,</span> <span class="hl kwc">type</span><span class="hl std">=</span><span class="hl str">&quot;class&quot;</span><span class="hl std">)</span>
                  <span class="hl kwa">if</span><span class="hl std">(</span><span class="hl opt">!</span><span class="hl kwd">is.null</span><span class="hl std">(submodels)) {</span>
                    <span class="hl std">tmp</span> <span class="hl kwb">&lt;-</span> <span class="hl std">out</span>
                    <span class="hl std">out</span> <span class="hl kwb">&lt;-</span> <span class="hl kwd">vector</span><span class="hl std">(</span><span class="hl kwc">mode</span> <span class="hl std">=</span> <span class="hl str">&quot;list&quot;</span><span class="hl std">,</span> <span class="hl kwc">length</span> <span class="hl std">=</span> <span class="hl kwd">nrow</span><span class="hl std">(submodels)</span> <span class="hl opt">+</span> <span class="hl num">1</span><span class="hl std">)</span>
                    <span class="hl std">out[[</span><span class="hl num">1</span><span class="hl std">]]</span> <span class="hl kwb">&lt;-</span> <span class="hl std">tmp</span>

                    <span class="hl kwa">for</span><span class="hl std">(j</span> <span class="hl kwa">in</span> <span class="hl kwd">seq</span><span class="hl std">(</span><span class="hl kwc">along</span> <span class="hl std">= submodels</span><span class="hl opt">$</span><span class="hl std">nIter)) {</span>
                      <span class="hl std">out[[j</span><span class="hl opt">+</span><span class="hl num">1</span><span class="hl std">]]</span> <span class="hl kwb">&lt;-</span> <span class="hl kwd">predict</span><span class="hl std">(modelFit,</span>
                                            <span class="hl std">newdata,</span>
                                            <span class="hl kwc">nIter</span> <span class="hl std">= submodels</span><span class="hl opt">$</span><span class="hl std">nIter[j])</span>
                    <span class="hl std">}</span>
                  <span class="hl std">}</span>
                  <span class="hl std">out</span>
                <span class="hl std">},</span>
                <span class="hl kwc">prob</span> <span class="hl std">=</span> <span class="hl kwa">NULL</span><span class="hl std">,</span>
                <span class="hl kwc">sort</span> <span class="hl std">=</span> <span class="hl kwa">function</span><span class="hl std">(</span><span class="hl kwc">x</span><span class="hl std">) x)</span>
</pre></div>
</div></div>

<p>Should you care about this? Let's tune the model over the same data set used for the SVM model above and see how long it takes:
</p>


<div class="chunk" id="custom_lb_mod"><div class="rcode"><div class="source"><pre class="knitr r"><span class="hl kwd">set.seed</span><span class="hl std">(</span><span class="hl num">825</span><span class="hl std">)</span>
<span class="hl std">lb1</span> <span class="hl kwb">&lt;-</span> <span class="hl kwd">system.time</span><span class="hl std">(</span><span class="hl kwd">train</span><span class="hl std">(Class</span> <span class="hl opt">~</span> <span class="hl std">.,</span> <span class="hl kwc">data</span> <span class="hl std">= training,</span>
                         <span class="hl kwc">method</span> <span class="hl std">= lbFuncs,</span>
                         <span class="hl kwc">tuneLength</span> <span class="hl std">=</span> <span class="hl num">3</span><span class="hl std">,</span>
                         <span class="hl kwc">trControl</span> <span class="hl std">= fitControl))</span>
</pre></div>
<div class="error"><pre class="knitr r">Error in eval(expr, envir, enclos): object 'training' not found
</pre></div>
<div class="output"><pre class="knitr r">Timing stopped at: 0 0 0 
</pre></div>
<div class="source"><pre class="knitr r"><span class="hl std">lb1</span>
</pre></div>
<div class="error"><pre class="knitr r">Error in eval(expr, envir, enclos): object 'lb1' not found
</pre></div>
<div class="source"><pre class="knitr r"><span class="hl com">## Now get rid of the submodel parts</span>
<span class="hl std">lbFuncs2</span> <span class="hl kwb">&lt;-</span> <span class="hl std">lbFuncs</span>
<span class="hl std">lbFuncs2</span><span class="hl opt">$</span><span class="hl std">predict</span> <span class="hl kwb">=</span> <span class="hl kwa">function</span><span class="hl std">(</span><span class="hl kwc">modelFit</span><span class="hl std">,</span> <span class="hl kwc">newdata</span><span class="hl std">,</span> <span class="hl kwc">preProc</span> <span class="hl std">=</span> <span class="hl kwa">NULL</span><span class="hl std">,</span> <span class="hl kwc">submodels</span> <span class="hl std">=</span> <span class="hl kwa">NULL</span><span class="hl std">)</span>
  <span class="hl kwd">predict</span><span class="hl std">(modelFit, newdata,</span> <span class="hl kwc">type</span><span class="hl std">=</span><span class="hl str">&quot;class&quot;</span><span class="hl std">)</span>
<span class="hl std">lbFuncs2</span><span class="hl opt">$</span><span class="hl std">loop</span> <span class="hl kwb">&lt;-</span> <span class="hl kwa">NULL</span>

<span class="hl kwd">set.seed</span><span class="hl std">(</span><span class="hl num">825</span><span class="hl std">)</span>
<span class="hl std">lb2</span> <span class="hl kwb">&lt;-</span> <span class="hl kwd">system.time</span><span class="hl std">(</span><span class="hl kwd">train</span><span class="hl std">(Class</span> <span class="hl opt">~</span> <span class="hl std">.,</span> <span class="hl kwc">data</span> <span class="hl std">= training,</span>
                         <span class="hl kwc">method</span> <span class="hl std">= lbFuncs2,</span>
                         <span class="hl kwc">tuneLength</span> <span class="hl std">=</span> <span class="hl num">3</span><span class="hl std">,</span>
                         <span class="hl kwc">trControl</span> <span class="hl std">= fitControl))</span>
</pre></div>
<div class="error"><pre class="knitr r">Error in eval(expr, envir, enclos): object 'training' not found
</pre></div>
<div class="output"><pre class="knitr r">Timing stopped at: 0 0 0 
</pre></div>
<div class="source"><pre class="knitr r"><span class="hl std">lb2</span>
</pre></div>
<div class="error"><pre class="knitr r">Error in eval(expr, envir, enclos): object 'lb2' not found
</pre></div>
</div></div>




































